Datacenter-on-Chip Architectures: Tera-scale Opportunities and Challenges in Intel's Manufacturing Environment

We have entered an era of chip multiprocessor (CMP) platforms, where performance is delivered with the integration of more and more cores on a die. Tera-scale CMP architectures, consisting of several tens of physical cores and hundreds of hardware threads, are highly suitable for throughput computing especially in the server market place. In this paper, we start by highlighting tera-scale potential in datacenter environments. We show how a multi-tier datacenter workload that required tens (to hundreds) of platforms in the past can potentially map on to one (or a few) single-socket tera-scale CMP platforms running Virtual Machines (VMs) and thereby creating Datacenter-onChip (DoC) architectures. Having introduced tera-scale DoC architectures, we then describe key challenges involved in providing high degrees of performance, scalability, and adaptability. Performance and scalability challenges point to the need for efficient handling of cache/memory/IO requirements when a large number of cores are actively running many workloads. Adaptability challenges highlight the need for dynamically allocating cache, memory, and I/O resources amongst the simultaneously running VMs in order to enable Quality of Service (QoS). To address scalability and adaptability challenges, we then propose and evaluate important tera-scale architectural features: (a) hierarchy of shared caches and large DRAM caches for better cache/memory scalability and performance, and (b) cache/memory QoS techniques to form Virtual Platform Architectures (VPAs). Based on a detailed evaluation, we show that these architectural features are highly beneficial for DoC tera-scale architectures.